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SUMMARY We investigate the secret key agreement from 
correlated Gaussian sources in which the legitimate parties can 
use the public communication with limited rate. For the class 
of protocols with the one-way public communication, we show a 
closed form expression of the optimal trade-off between the rate 
of key generation and the rate of the public communication. Our 
results clarify an essential difference between the key agreement 
from discrete sources and that from continuous sources. 
key words: Bin Coding, Gaussian Sources, Privacy Amplifica- 
tion, Quantization, Rate Limited Public Communication, Secret 
Key Agreement 

1. Introduction 

Key agreement is one of the most important prob- 
lems in the cryptography, and it has been extensively 
studied in the information theory for discrete sources 
(e.g. [1], [11], [12]) since the problem formulation by 
Maurer [19]. Recently, the confidential message trans- 
mission [10], [26] in the MIMO wireless communication 
has attracted considerable attention as a practical prob- 
lem setting (e.g. [7], [17], [18]). Although the key agree- 
ment in the context of the wireless communication has 
also attracted considerable attention recently [6], the 
key agreement from analog sources has not been stud- 
ied sufficiently compared to the confidential message 
transmission. As a fundamental case of the key agree- 
ment from analog sources, we consider the key agree- 
ment from correlated Gaussian sources in this paper. 
More specifically, we consider the problem in which the 
legitimate parties, Alice and Bob, and an eavesdrop- 
per, Eve, have correlated Gaussian sources respectively, 
and Alice and Bob share a secret key from their sources 
by using the public communication. Recently, the key 
agreement from Gaussian sources has attracted consid- 
erable attention in the context of the quantum key dis- 
tribution [13], which is also a motivation to investigate 
the present problem. 

Typically, the first step of the key agreement pro- 
tocol from analog sources is the quantization of the 
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sources. In the literatures (e.g. sec [2], [3], [6]), the 
scalar quantizer is used, i.e., the observed source is 
quantized in each time instant. Using the finer quanti- 
zation, we can expect the higher key rate in the proto- 
col, where the key rate is the ratio between the length 
of the shared key and the block length of the sources 
that are used in the protocol. However, there is a prob- 
lem such that the finer quantization might increase the 
rate of the public communication in the protocol. Al- 
though the public communication is usually regarded 
as a cheap resource in the context of the key agreement 
problem, it is limited by a certain amount in prac- 
tice. Therefore, we consider the key agreement pro- 
tocols with the rate limited public communication in 
this paper. The purpose of this paper is to clarify the 
optimal trade-off between the key rate and the pub- 
lic communication rate of the key agreement protocol 
from Gaussian sources. It should be emphasized that 
we consider the optimal trade-off among the protocols 
with not only the scalar quantizer but also the vector 
quantizer. 

The key agreement by rate limited public commu- 
nication was first studied by Csiszar and Narayan for 
discrete sources [11]. For the class of protocols with the 
one-way public communication, they characterized the 
optimal trade-off between the key rate and the public 
communication rate in terms of information theoretic 
quantities, i.e., they derived the so-called single letter 
characterization. However, there are two difficulties to 
extend their result to the Gaussian sources. 

First, the direct part of the proof in [11] heavily 
relics on the finiteness of the alphabets of the sources, 
and cannot be applied to continuous sources. We show 
the direct part by using a method that is similar to the 
information spectrum approach [14]. 

Second, although the converse part of Csiszar and 
Narayan's characterization can be easily extended to 
continuous sources, the characterization is not com- 
putable because the characterization involves auxiliary 
random variables and the ranges of those random vari- 
ables arc unbounded for continuous sources. In this pa- 
per, we show that Gaussian auxiliary random variables 
are sufficient, and we derive a closed form expression 
of the optimal trade-off. A key tool in the derivation 
of the closed form expression is the entropy power in- 
equality [9] , which has been applied to solve the Gaus- 
sian multitcrminal problems in the literatures [5], [16], 
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[22], [23]. 

There is another work that is related to this paper. 
Nitinawarat studied the problem in which Alice and 
Bob have correlated Gaussian sources and they share a 
secret key from their sources by the public communica- 
tion [21]. The problem formulation in this paper can be 
regarded as a generalization of [21] to the case in which 
Eve also has a Gaussian source. It should be noted 
that [21] considered the key agreement with the rate 
limited quantization instead of the rate limited public 
communication. 

The rest of this paper is organized as follows. In 
Section 2, we formulate the problem treated in this pa- 
per. Main results and the outlines of their proofs are 
presented in Section 3. Conclusions are discussed in 
Section 4, and the details of the proofs are presented in 
Appendices. 

2. Preliminaries 

Let X, Y, and Z be zero- mean correlated Gaussian 
sources on the set of real numbers K respectively. Then, 
let X n , Y n , and Z n be i.i.d. copies of X, Y, and Z re- 
spectively. We assume that Alice, Bob, and Eve know 
the covariance matrix of (X, Y, Z). Throughout the pa- 
per, upper case letters indicate random variables, and 
the corresponding lower case letters indicate their re- 
alizations. We use the same notations as [9] for the 
entropy, the mutual information, etc.. 

Although Alice and Bob can use the public com- 
munication interactively in general, we concentrate on 
the class of key agreement protocols in which only Alice 
sends a message to Bob over the public channel^. First, 
Alice computes the message C n from X n and sends the 
message to Bob over the public channel. Then, she also 
compute the key S n . Bob computes the key S' n from 
Y n and C n . 

The error probability of the protocol is defined by 

e n := Pr{S„ + S'J. 

The security of the protocol is measured by the quantity 

v n :=\og\S n \-H(S n \C n ,Z n ), (1) 

where S n is the range of the key S n , and |<S n | indicates 
the cardinality of the set S n . 

In this paper, we are interested in the trade-off be- 
tween the public communication rate R p and the key 
rate Rk. The rate pair (Rp, Rk) is defined to be achiev- 
able if there exists a sequence of protocols satisfying 



lim e n = 0, 

n— >oo 

lim v n = 0, 



(2) 
(3) 



^It should be noted that the results in this paper is valid 
for the class of key agreement protocols in which only Bob 
sends a message to Alice. 



lim sup - log | C n | < Rp, 
liminf — log |<S„| > R k , 



(4) 
(5) 



where C n is the range of the message C„ transmitted 
over the public channel. Then, the achievable rate re- 
gion is defined as 

TZ(X,Y,Z) := {{R P , Rk) ■ (R P , Rk) is achievable}. 

The purpose of this paper is to derive a closed form 
expression of the rate region TZ(X, Y, Z). 

3. Main Results 

3.1 Statement of Results 

In this section, we show a closed form expression of the 
rate region 1Z(X, Y, Z), which will be proved in the next 
section. Let 



E = 



be the covariance matrix of (X, Y, Z) . Throughout the 
paper, we assume that E is positive definite and H xy ^ 
because the key agreement is obviously impossible if 
Exy = 0. Then, we can write (see Appendix C.l) 

X = K XZ Z + W 1 , (6) 
Wi (7) 
(8) 



Ea; 


E;cy 


E X z 


Eya; 


Yjy 


Ey 2 


E zx 


^zy 





Y = K yx X 



K yz Z 



= (K yx K xz + K yz )Z + K yx Wi + W 2 , 

where W\ and W2 are zero-mean Gaussian random vari- 
ables independent of each other, W\ is independent of 
Z, and W-2 is independent of (X,Z). The coefficients 
are given by 



and 



[ Kyz K yx ] — [ Yiyz ^ yx ] 



E 2 



Furthermore, we also have 
En/! = Ej.| 2 = E x — K x 

EV^2 ^y\xZ Ey Ky X Yj X y Ky Z Y!l Z y, 



^XZ^ZX , 



^yz^z ^zy 



(9) 
(10) 

(11) 



where E^ and E^ 2 are the variances of W± and W2 
respectively, E^u is the conditional variance of X given 
Z, ^ y \xz and E y | z are the conditional variances of Y 
given (X, Z) and Z respectively. 
For R p > 0, let 

Rk(Rp) := sup{R k : (R k ,R P ) & U(X,Y,Z)}. 
Before investigating the rate region, we present well 
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known upper bound on the function R k (R p ), which was 
shown for the discrete sources in [1], [19], and can be 
shown in a similar manner for continuous sources. 

Proposition 1 ([1], [19]) For any R p > 0, we have 
R k (R p )<I(X;Y\Z) = Jlog^. (12) 

Remark 2 Although we will concentrate on the class 
of key agreement protocols in which only Alice sends 
a message to Bob over the public channel, the upper 
bound in Proposition 1 is still valid even if we consider 
the class of protocols in which Alice and Bob sends 
messages interactively. 

Remark 3 In Eq. (12), we use the fact that (X, Y, Z) 
are Gaussian only to derive the right equality, and the 
left inequality holds for any continuous sources. 

We first consider the case such that the sources are 
degraded, i.e., they satisfy the Markov chain 

X++Y++Z. 

Theorem 4 Suppose that (X, Y, Z) are degraded. 
Then, we have 

TZ(X,Y,Z) = SjRp,R k ) : 

R k < - log L .(is) 

y\xz J 

As we can find from the above theorem, the function 
Rk(Rp) is concave and monotonically increasing, and it 
converges to the upper bound in Proposition 1 as R p 
goes to infinity. 

Remark 5 When (X, Y, Z) are discrete sources and 
are degraded, it is known [1], [11] that 

R k (R p ) = I(X;Y\Z). 

for any R p > H(X\Y). Furthermore for Rp > H(X\Y), 
R k (R P ) can be achieved by the combination of the 
Slcpian-Wolf coding [24] and the privacy amplification 
(e.g. see [4]), and the quantization of Alice's source is 
not necessary. On the other hand, Theorem 4 implies 

R k (Rp) <I(X;Y\Z) 

for any finite R p . This fact suggests an essential differ- 
ence between the key agreement from discrete sources 
and that from continuous sources. 

When we consider the protocol with only one-way 
public communication, note that the error probabil- 
ity e n and the security parameter v n only depend on 
the marginal densities p{x, y) and p(x, z) respectively. 



More precisely, let (X, Y, Z) be random variables such 
that the marginal densities of (X,Y) and (X,Y), and 
those of (X, Z) and (X, Z) coincide respectively. Then 
we have 

TZ(X,Y,Z)=TZ(X,Y,Z). 

By using this fact and the following lemma, the proof 
of which will be presented in Appendix C.2, we can 
always reduce the general case to the degraded case. 

Lemma 6 If the square of the correlation coefficient of 
(X, Y) is larger than that of (X, Z), i.e., 

then there exist jointly Gaussian sources (X, Y, Z) such 
that 

X^Y^Z (15) 

is satisfied, and that the marginal densities of (X, Y) 
and (X,Y), and those of (X,Z) and (X,Z) coincide 
respectively. 

On the other hand, if the square of the correlation 
coefficient of (X, Y) is smaller than or equal to that of 
(A, Z), i.e., 

^xy^y 1 '^x 1 — ^xz^z ) (16) 

then there exist (not necessarily Gaussian) sources 
(X,Y,Z) such that 

loZor (17) 

is satisfied, and that the marginal densities of (X, Y) 
and (X,Y), and those of (X,Z) and (X,Z) coincide 
respectively. 

When there are jointly Gaussian sources (X, Y, Z) sat- 
isfying Eq. (15), we can compute the region by us- 
ing Theorem 4. On the other hand, when there are 
(X, Y, Z) satisfying Eq. (17), Proposition 1 implies 
R k (Rp) = for any Rp > 0. 

3.2 Proof of Theorem 4 

3.2.1 Converse Part 

In order to prove the converse part, we need the fol- 
lowing proposition and corollary. The proposition was 
shown for discrete sources in [11, Theorem 2.6], and it 
can be shown almost in the same manner for continuous 
sources. 

Proposition 7 ([11]) Suppose that a rate pair 
(R p ,R k ) is included in 7Z(X, Y,Z). Then, there exist 
auxiliary random variables U and V satisfying 

Rp > I(U;X\Y), (18) 
R k <I(U;Y\V)-I(U;Z\V), (19) 
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and the Markov chain 

v o u o x o 



(20) 



For degraded sources, we can simplify the above propo- 
sition, which will be shown in Appendix B. 

Corollary 8 Suppose that (X,Y,Z) is degraded, i.e., 
X Y <r> Z. If {R p ,Rk) e 7£(X,Y,Z), then there 
exists an auxiliary random variable {/ satisfying 



i? p > 7(C/;X|r), 

i? fc < /(u-;y|z), 

and the Markov chain 

u <-> x <-> r <-> z. 



(21) 
(22) 

(23) 



Proof of Converse Part) 

Part of ideas of the following proof are borrowed 
from [25] with proper modifications. In the following, 
h(-) and h(-\-) designate the differential entropy and the 
conditional differential entropy respectively [9] . 

We will show 



-2R k 



Rk- 



(24) 



J y\xz 



Then, by solving the inequality with respect to Rk, we 
have the converse part of Theorem 4. 

For any auxiliary random variable U satisfying 
Eq. (23), by a straightforward calculation, we have 

h(Y\Z) - I(U;Y\Z) 
= h(Y\U,Z) 

= h(K yx X + K yz Z + W 2 \U,Z) 
= h(K yx X + W 2 \U,Z). 

Then, by using the conditional version of the entropy 
power inequality (EPI) [5] , we have 

cxp [2h{K yx X + W 2 \U,Z)} 
> exp [2h(K yx X\U, Z)\ + exp [2h{W 2 )\ 
= K 2 yx cxp [2h(X\U, Z)] + cxp [2h(W 2 )] 
= K* x exp[-2I(U;X\Z) + 2h(X\Z)] 

+ exp [2h(W 2 )\ 
= K 2 yx c^>[-2I{U;X\Z) + 2h{W 1 )] 

+ exp[2h(W 2 )]. 

Thus, wc have 



I(U;X\Z)-I(U;Y\Z) 
1 
2 



> -log[^exp{2/ l (^ 1 )}] 



-i log [exp{2h(Y\Z) - 2I(U; Y\Z)} 
- exp{2/i(W 2 )}] - I(U; Y\Z). (25) 
From Eqs. (10) and (8), we can find that the variances 



of W 2 and K yx W\ are Y, y \ xz and K% x E Wl = T, y \ z ~T, y \ xz 
respectively. Thus, we can rewrite the right hand side 
of Eq. (25) as 



I(U;Y\Z). 



^y\z 

Since the function 

-log S »l«- S «l** _ a 

is monotonically increasing for < a < I(X; Y\Z) and 

I(U;X\Z)-I(U;Y\Z) = I(U; X\Y) 

for (U, X, Y, Z) satisfying Eq. (23), Corollary 8 implies 
Eq. (24). □ 

3.2.2 Direct Part 

In order to prove the direct part, we need the following 
proposition, which can be regarded as a generalization 
of [11, Theorem 2.6] to continuous sources^. We show 
a proof in Appendix A because the proof of [11, Theo- 
rem 2.6] heavily relies on the finiteness of the alphabets 
and its generalization to continuous sources seems non- 
trivial. It should be noted that the following proposi- 
tion holds for non-degraded case. 

Proposition 9 For an auxiliary random variable U 
satisfying the Markov chain 

U^X^{Y,Z), (26) 

let (R p ,Rk) be a rate pair such that 

R p > I(U;X)-I(U;Y), 
Rk < I(U;Y)-I(U;Z). 

Then, we have (R p , R k ) € TZ(X, Y, Z). 

Note that 

I{U;X\Y) = I(U;X)-I(U;Y), 
I(U;Y\Z) = I(U;Y)-I(U;Z) 

for degraded sources. 
Proof of Direct Part) 

Let W be a zero mean Gaussian random variable, 
and U = X + W . Since the (conditional version of) 
entropy power inequality holds with equality for Gaus- 
sian random variables [5], [9], Eq. (25) in the converse 
part holds with equality, i.e., we have 



^ Although [11, Theorem 2.6] involves two auxiliary ran- 
dom variables, we only show the version with only one aux- 
iliary random variable because one of the auxiliary random 
variables in [11, Theorem 2.6] is not needed to show Theo- 
rem 4. 
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I(U;X\Y) 

= I(U;X\Z)-I{U;Y\Z) 
1 



^y\z ^y\xz 



9 ^ y , 



I(U;Y\Z). 



Thus, by setting I(U;Y\Z) = R k and I(U;X\Y) = 
R p , by solving the equality with respect to Rk, and by 
adjusting the variance of W, we find that any (R p , Rk) 
satisfying the equality 



i? fe = ilog^^ 
is achievable. 



-2R V 



-2R V 



J y\xz 



4. Conclusions and Discussions 



We investigated the secret key agreement from Gaus- 
sian sources by rate limited public communication. For 
the class of protocols with the one-way public commu- 
nication, we derived a closed form expression of the 
optimal trade-off between the key rate and the pub- 
lic communication rate. The optimal trade-off for the 
class of protocols with the two-way public communica- 
tion remains unsolved and investigating it is a future 
research agenda. 

Our result suggested an essential difference be- 
tween the key agreement from discrete sources and 
that from continuous sources (Remark 5). For discrete 
sources, if the public communication rate is larger than 
H(X\Y), the upper bound can be achieved without 
quantization. On the other hand for Gaussian sources, 
the upper bound cannot be achieved for any finite pub- 
lic communication rate. 

The problem formulation treated in this paper can 
be regarded as Gaussian version of the source type 
model [1]. We can also consider Gaussian version of 
the channel type model. In such a model, Alice can 
send a signal, with power constraint, to Bob and Eve 
over Gaussian channels. In addition to the Gaussian 
channel, Alice and Bob can use the public communica- 
tion with limited rate. For the class of protocols with 
the forward public communication^ , by a slight modifi- 
cation of the proof of [1, Theorem 2], we can show that 
the suprcmum of achievable key rates coincides with 
the secrecy capacity of the Gaussian wiretap channel 
[16] no matter what the limitation of the public com- 
munication rate. 

Appendix A: Proof of Proposition 9 

For arbitrarily fixed auxiliary random variable satisfy- 
ing Eq. (26), we show that the rate pair 

^The forward (backward) public communication means 
that only Alice (Bob) sends a public message to Bob (Alice) . 
It should be noted that the forward public communication 
and the backward public communication make significant 
difference for the channel type model. 



Rp = I(U; X) 
R k = I(U;Y)- 



I(U;Y) 
I{U;Z)- 



-47, 
67 



is achievable for any 7 > 0. Instead of showing 
the achievability for the security criterion defined by 
Eq. (1), we show the achievability for the security cri- 
terion defined by 



M» := J P(z n ) 

\\P SnCnlz 4-,^z n ) - P 5 J-)P Cnl M^\\dz n , 

where Pg is the uniform distribution on the key alpha- 
□ bet S n , and || • || is the variational distance [9]. More 
precisely, we show that there exists a sequence of pro- 
tocols satisfying Eqs. (2), (4) and (5) and fi n converges 
to exponentially. Then, by using [20, Lemma 3], we 
can also show that v n also converges to 0. 

Our protocol roughly consists of three steps: the 
quantization, the bin coding [9], and the privacy ampli- 
fication. First, Alice quantizes her source by a function 
g n : R™ — > Q n C R™. We use the auxiliary random 
variable U for quantization almost in a similar manner 
as the Wyner-Ziv problem [27]. After the quantization, 
she sends the bin index C n = 4> n (gn{X n )) by a func- 
tion <j) n : Q n — > C„ . Bob decodes the index and his 
source by a function ip n : C n x R" — > Q n . Note that the 
public communication rate R p must be large enough 
so that Bob can recover the quantized source by us- 
ing his source Y n as side-information at the decoder. 
Finally, they obtain keys S n = f n (g n (X n )) and S' n = 
fn{^n(4>n{C n ),Y n )) by a function /„ : Q n -> S n re- 
spectively. Existence of functions {(g n , 4>n, ipn, fn)}n°=i 
satisfying Eqs. (2), (4) and (5) and fx n — > are guar- 
anteed by the following lemmas. Lemma 10 is the so- 
called Markov lemma, the proof of which will be omit- 
ted (e.g. see [15]). In order to upper bound the error 
probability e„, we need Lemma 11, which appears in 
the course of deriving the general formula of the Wyner- 
Ziv problem [15]. We also omit the proof. In order 
to upper bound the security parameter /x„, we need 
Lemma 12, which is an extension of the privacy ampli- 
fication lemma shown in [20, Lemma 4]. Since we ap- 
ply the privacy amplification to the (vector) quantized 
source in our protocol, we need Lemma 12. The proof 
of Lemma 12 is the most difficult part of the proof of 
Proposition 9, and it will be proved in the next section. 

For a fixed auxiliary random variable U satisfying 
Eq. (26) and t,a,0 € R, let 



■An 



{u ^ x n ): i log P(^l <t 

v ' n p{u n ) ~ 

(«^):ilog^«>a 
n p(y n ) 

(u n ,x n ,z n ) : -log ^r^'nT^ - £ 
n p(x n \z n ) 
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Lemma 10 For any t,a, (3 £ R, there exists a function 
g n : R" -» Q n such that 

Pr{(g n (X n ),Y n ) £ A n or (g n (X n ),X n , Z n ) £ B n ] 
< 2^/5~ n + Vr{{U n ,X n ) £ T n } + cxp{-|Q„| e - t "}, 

where 

S n := Pr{([/",F") £ A n or (U n ,X n ,Z n ) £ B n }. 



Lemma 11 For any function g r , 
R, there exist functions <p n : Q n 
R™ -> Qn such that 



J™ -> Q„ and a G 
C„ and tp n ■ C n x 



Pr{. gn (X")^Vn(0„(ff„(X")),F")} 



< 



Ignj 
|C„| 



" a " + Pr{(g n (X n ), Y n ) £ A n }. 



Lemma 12 For any functions g n '■ R™ — > Q n , 4>n 
Q n — > C n , and £ R, there exists a function /„ 
Q n — > 5 n such that 



M« < ^|5„||C„|e-^ + 2Pr{( ffn (X n ),X n ,Z n ) £ B„}. 

Note that only the cardinality of C n appears in the up- 
per bound on fi n no matter the structure of a specific 
function <p n . However the functions f n realizing the 
upper bound depend on the structure of <p n . 
From Lemmas 10, 11, and 12, by setting 



Qn\ 


= exp{n(I(U; X) 4 


-2 7 )}, 




\Cn\ 


= exp{n(I(£/;X) - 


■I(U;Y)-[ 


-4 7 )}, 


\Sn\ 


= exp{n(I(U; Y) - 


I(U;Z)- 


67)}, 


t 


= I(U;X)+j, 






a 


= I{U;Y)- 1 , 








= I{U-X\Z)- 1 , 







and by noting that I(U\X\Z) = I(U;X) - I(U;Z), 
we obtain a sequence of protocols satisfying Eqs. (2), 
(4) and (5) and fj, n exponentially^ converges to 0. By 
using [20, Lemma 3], we can show that Eq. (3) is also 
satisfied. 

Finally, by taking a sequence {7^} such that 71 > 
72 > ■ ■ • > and 7j — s- (i — > 00) instead of 7 > 0, and 
by using the diagonalization argument [14], we have 
Proposition 9. □ 

A.l Proof of Lemma 12 

In the following, we use the notation 

f-^s) ~{u n €Q n :f n (u n ) = s}. 

The sets ^~ 1 (c) for c £ C n and g~ l {u n ) for u n £ Q n 
arc defined in similar manners. Furthermore, for a set 

•We use the Chernoff bound (e.g. see [9]) instead 
of the Chebyshev inequality to upper bound 5„ and 
Pr{(U",X n ) £ T n }. 



A C Q n , we denote g n l {A) := l) uneA g n 1 (u n ). For a 
set B c 1™, we define 

P x „ lZn (B\z n ) := Pr{X n £ B\Z n = z n }. 

In this section, it should be also noted that summations 
are taken over the range of the indices unless otherwise 
specified. 

For fixed z n £ R™, let 



{x n : ( 9n (x n ),x n ,z n )eB n }, 



and B c z n be the complement of B z n in R™. Then, by a 
straightforward calculation, we have 

\\Ps„c n \zA-,-\z n )-PsA-)Pc n \zA-\z n )\\ 
= Y,\ P Sr>c n \zAs,c\z n )-PsAs)P CnlZn (c\z n )\ 

s,c 

= E iPx^zAg-Hf-'is) n <t>-\c))nB^\z n ) 

-P Sii ( S )P xre | Zre (.g- 1 (0- 1 ( C ))n^|z™) 

+P X niz4g- 1 (f- 1 (s) n <p-\c)) n Bl n \z n ) 
-Ps n (s)P Xn]zn {g-\ct>-\c)) n B c zn \z n )\ 
< E \Pxni Z n(g-\f- '(s) n 4>-\c))nB^\z n ) 

s,c 

-P5 n ( s )p^i^( 5 - 1 (0- 1 ( c ))n6 z , 1 |z»)| 
+ E \Px^{9n\fn\s) n ^(c)) nBS»|0 

-PgJ^P^l^fc 1 ^- 1 ^)) n ^„|z")| 
= E l^i^feH/n \s) n c 1 ^)) ns z „|z") 

s,c 

-P 5f> ( S )P X n lZ n (g- 1 ( C )) n £ z „ |z")| 

+2P x „| Z „(^„|z"), (A-l) 

where we used the triangle inequality. By using the 
Cauchy-Schwarz inequality, the first term of Eq. (A- 1) 
is upper bounded by 

|<S n [|C n | E \Px^(gn l {f-\s) n c'Cc)) n B zn \z n ) 



Pg„( s )p x „i^( 5 - 1 (0- 1 ( c ))n^|z»)| 2 



1/2 



(A 2) 



Furthermore, we can rewrite the inside of the square 
root of Eq. (A- 2) as 

E iPx^zAgnHfn 1 ^) n ^(c)) n B z „|^) 
-^5„W^i^(.9 I ; 1 (C 1 (c))n6 2 „|z")| 2 
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where we used the facts P§ n (s) = and 

E^ls-fo^UnH*) n ^(c)) n s 2 ,|z") 

s 

= Px»|^(ffnW(c))nB,»|* n ). 
We can rewrite the first term of Eq. (A- 3) as 



(A- 3) 



,n\2 



= E E 

s,c u » j u»e/- 1 (s)n^ 1 (c) 

Px»|z«fc 1 («")nB z „|z")P x „| Z „( ff - 1 (w")nB z „|^) 

= E E <5 / n («"),/™(«' 1 ) 

S ' C U n jU^-G^n 1 ( c ) 

Pxn|^»(fl^(« n )nB*»IOi'x-i^(fln 1 (fi n )nB»»|« n ) ) 

(A 4) 

where 5 a) & = 1 if a = 6 and # aj & = otherwise. In 
a similar manner, we can rewrite the second term of 
Eq. (A- 3) as 

E ij^Js^M^WM) n BlAz n Y 

c ' n ' 

= E^j E 

P x ^zA9n\n n )nB z Az n )Px^{g-\u n )nB zn \z n ). 

(A- 5) 

Let J-'n be a universal hash family of functions from 
Q n to S n [8], i.e., 

P Fn ({/„ G -P„ : /n(« n ) = /»(£")}) < t^-t 

for any distinct u™ and u™, where Pp„ is the uniform 
distribution on J- n . Combining Eqs. (A-3)-(A-5), wc 
can evaluate the inside of the square root of Eq. (A- 2) 
averaged over the random choice of /„ as follows: 

%. ^P X ^(3nHfnHs) n ^(C)) n £ 2 „|z") 2 
_ s,c 

- E j^^l^fcHC 1 ^)) H B z « \ z -f 
= E E E 6 fn - ro 

Pa-iz" V) n £ 2 ~ |z")Px-|Z" (^(fi") n £ 2 „ |z") 



^E E 

c u"e0„ 1 (c) 

- E 

tt"GQn 

Pa-|Z" (<?~ V") n B a » |z")Px"|Z" fc 1 ^) n B a »|z") 

(A 6) 

Since 

p{x n \z n ) <p(x n \u n ,z n )e-P n 
for (it™, x n , z n ) £ B n , Eq. (A- 6) is upper bounded by 

E iW»(s»V)nB»»|* B ) 
Pxn| t /n^te 1 K)nS^K,^)e-^ 

< E ^»|z»(ffn 1 (« n )nB,»|« T, )e-' Sn 
u»eS„ 

< e- pn . (A- 7) 

Since the square root function ^f- is concave, by com- 
bining Eqs. (A-3)-(A-7), Eq. (A- 2) averaged over f n 
is upper bounded by ^/|<5> n | |C„|e _,9n . By substituting 
this upper bound into Eq. (A- 1), by taking the average 
over z n £ R™, and by using the concavity of y^, wc 
have 



%„[Mn] < ^I^HCnle-/ 3 " 
+21>r{(g n (X n ),X n ,Z n )?B n }. 

Thus, there exists at least one /„ £ T n satisfying the 
statement of the lemma. □ 

Appendix B: Proof of Corollary 8 

For any (U,V) satisfying Eqs. (18) and (20), wc have 

I(V;Y)+I(U;Y\V) = I(U,V;Y) 

= I(U;Y)+I(V;Y\U) 
= I(U;Y), 

where the last equality follows from the fact that V,U, 
and Y form a Markov chain. Similarly, we have 

I(V;Z) + I(U-Z\V) = I{U-Z). 
Since wc have 

I(V;Y)>I(V;Z) 
for degraded sources, we have 

I(U; Y\V) - I(U; Z\V) < I(U; Y) - I(U; Z), 
which implies the assertion of the corollary. □ 

Appendix C: Miscellaneous Facts 

For reader's convenience, we review some basic facts on 
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jointly Gaussian random variables. 
C.l Derivations of Eqs.(6)-(8) 

The probability density function of zero-mean Gaussian 
random vector X is uniquely determined by its covari- 
ance matrix Ex- Furthermore, for any non-degenerate 
matrix A, the covariance matrix of the Gaussian ran- 
dom vector X' = AX is given by E X ' = AZ^_A T . On 
the other hand, any non-degenerate symmetric matrix 
of the form 



References 



M 



A B 
B T C 



can be decomposed as 
M = 



I 

B T A~ l I 



A 

C-B T A- 1 B 



I A~ X B 
I 



By using these facts, we can derive Eqs.(6)-(8). 
C.2 Proof of Lemma 6 

By using facts in Appendix C.l, we can write 

Y = T, xy Y, x 1 X + N y , 
Z = Z^Z^X + Nz, 

where N y and N z are Gaussian random variables that 
are independent of X and the variances arc E y — 
E^E" 1 and E z - E^E" 1 respectively. When Eq. (14) 
is satisfied, by setting X = X, Y = Y, and 

Z = E, 2 E-Jy + N 

for Gaussian random variable N with variance E z — 
E^ z E y E~y , we obtain jointly Gaussian sources satisfy- 
ing the assertion of the lemma, where N is independent 
of the other random variables. When Eq. (16) is satis- 
fied with strict inequality, we can obtain jointly Gaus- 
sian sources satisfying the assertion of the lemma in a 
similar manner. When Eq. (16) is satisfied with equal- 
ity, by setting X = X . Z = Z . and Y = Z xy Z~}Z, we 
obtain sources (not jointly Gaussian because the covari- 
ance matrix is degenerated) satisfying the assertion of 
the lemma. □ 
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